Approximation of stationary processes by hidden Markov models

We propose an algorithm for the construction of a Hidden Markov Model (HMM) of assigned complexity (number of states of the underlying Markov chain) which best approximates, in Kullback-Leibler divergence rate, a given stationary process. We establish, under mild conditions, the existence of the divergence rate between a stationary process and an HMM, and approximate it with a properly defined divergence between their Hankel matrices. The proposed three-step algorithm, based on the Nonnegative Matrix Factorization technique, realizes an HMM optimal with respect to the Hankel approximated criterion. A full theoretical analysis of the algorithm is given in the special case of Markov approximation.